Provenance as Data Mining: Combining File System Metadata with Content Analysis

Provenance describes how an object came to be in its present state. Thus, it describes the evolution of the object over time. Prior work on provenance has focussed on databases and the file system. The database or file system is enhanced or augmented in order to capture additional information about the historical evolution of document collections, and thus answer the provenance question. We address the question of provenance for unstructured information (i.e., document corpii from file systems) but without any enhancements to the file system. To provide a solution in this setting, we model the provenance problem in such a setting as a problem of data mining. We show that data mining can provide provenance information for repositories of unstructured information, including chains of historical evolution. Thus, we do not require any additions to the file system, and we can operate on legacy documents. Experimental results indicate a strong performance of our approach. 1 The Provenance problem The explosion of data of all forms in recent times has given rise to many research investigations, one of which is Provenance. Roughly speaking, Provenance is the study of (a) the origins and (b) the history of evolution of data. In other words, the Provenance question for a piece of data asks “How did this data originate, and how did it evolve into its current form?” The provenance question can be asked in unstructured (filesystem) environments, and in structured (database) environments. We will be concerned only with unstructured provenance in this paper. Concretely, given an evolving universe Ut of documents at time t, and a finite subset of documents {Di}1≤i≤M ⊂ UT from the universe of documents at a time T , the Provenance question is a specific question about the relationship of {Di} to Ut for t ≤ T . It asks: (i) Which documents in Ut for t ≤ T contributed to the evolution of documents {Di} ⊂ UT ? (ii) What are the first document(s) in the chain of ideas that culminated in the documents {Di}? (iii) When did the chain of documents that led to {Di} ⊂ Ut begin? In other words, what is the lowest value of t for which Ut contains a document which has contributed to the evolution of a document in {Di} ⊂ UT ? We shall refer to the first two questions as Where Provenance and the third question as When Provenance.